Generalization Performance Of Model Research Articles

G-YOLO: A YOLOv7-based target detection algorithm for lightweight hazardous chemical vehicles.

Hazardous chemical vehicles are specialized vehicles used for transporting flammable gases, medical waste, and liquid chemicals, among other dangerous chemical substances. During their transportation, there are risks of fire, explosion, and leakage of hazardous materials, posing serious threats to human safety and the environment. To mitigate these possible hazards and decrease their probability, this study proposes a lightweight object detection method for hazardous chemical vehicles based on the YOLOv7-tiny model.The method first introduces a lightweight feature extraction structure, E-GhostV2 network, into the trunk and neck of the model to achieve effective feature extraction while reducing the burden of the model. Additionally, the PConv is used in the model's backbone to effectively reduce redundant computations and memory access, thereby enhancing efficiency and feature extraction capabilities. Furthermore, to address the problem of performance degradation caused by overemphasizing high-quality samples, the model adopts the WIoU loss function, which balances the training effect of high-quality and low-quality samples, enhancing the model's robustness and generalization performance. Experimental results demonstrate that the improved model achieves satisfactory detection accuracy while reducing the number of model parameters, providing robust support for theoretical research and practical applications in the field of hazardous chemical vehicle object detection.

Sex classification from functional brain connectivity: Generalization to multiple datasets.

Machine learning (ML) approaches are increasingly being applied to neuroimaging data. Studies in neuroscience typically have to rely on a limited set of training data which may impair the generalizability of ML models. However, it is still unclear which kind of training sample is best suited to optimize generalization performance. In the present study, we systematically investigated the generalization performance of sex classification models trained on the parcelwise connectivity profile of either single samples or compound samples of two different sizes. Generalization performance was quantified in terms of mean across-sample classification accuracy and spatial consistency of accurately classifying parcels. Our results indicate that the generalization performance of parcelwise classifiers (pwCs) trained on single dataset samples is dependent on the specific test samples. Certain datasets seem to "match" in the sense that classifiers trained on a sample from one dataset achieved a high accuracy when tested on the respected other one and vice versa. The pwCs trained on the compound samples demonstrated overall highest generalization performance for all test samples, including one derived from a dataset not included in building the training samples. Thus, our results indicate that both a large sample size and a heterogeneous data composition of a training sample have a central role in achieving generalizable results.

The optimization of evaporation rate in graphene-water system by machine learning algorithm

Solar interfacial evaporation, as a novel practical freshwater production method, requires continuous research on how to improve the evaporation rates to increase water production. In this study, sets of data were obtained from molecule dynamics simulation and literature, in which the parameters included height, diameter, height–radius ratio, evaporation efficiency, and evaporation rate. Initially, the correlation between the four input parameters and the output of the evaporation rate was examined through traditional pairwise plots and Pearson correlation analysis, revealing weak correlations. Subsequently, the accuracy and generalization performance of the evaporation rate prediction models established by neural network and random forest were compared, with the latter demonstrating superior performance and reliability confirmed via random data extraction. Furthermore, the impact of different percentages (10%, 20%, and 30%) of the data on the model performance was explored, and the result indicated that the model performance is better when the test set is 20% and all the constructed model converge. Moreover, the mean absolute error and mean squared error of the evaporation rate prediction model for the three ratios were calculated to evaluate their performance. However, the relationship between the height- radius ratio and optimal evaporation rate was investigated using the enumeration method, and it was determined that the evaporation efficiency was optimal when the height–radius ratio was 6. Finally, the importance of height, diameter, height– radius ratio, and evaporation efficiency were calculated to optimize evaporator structure, increase evaporation rate, and facilitate the application of interfacial evaporation in solar desalination.

Prompt guidance query with cascaded constraint decoders for human–object interaction detection

AbstractHuman–object interaction (HOI) detection, which localises and recognises interactions between human and object, requires high‐level image and scene understanding. Recent methods for HOI detection typically utilise transformer‐based architecture to build unified future representation. However, these methods use random initial queries to predict interactive human–object pairs, leading to a lack of prior knowledge. Furthermore, most methods provide unified features to forecast interactions using conventional decoder structures, but they lack the ability to build efficient multi‐task representations. To address these problems, we propose a novel two‐stage HOI detector called PGCD, mainly consisting of prompt guidance query and cascaded constraint decoders. Firstly, the authors propose a novel prompt guidance query generation module (PGQ) to introduce the guidance‐semantic features. In PGQ, the authors build visual‐semantic transfer to obtain fuller semantic representations. In addition, a cascaded constraint decoder architecture (CD) with random masks is designed to build fine‐grained interaction features and improve the model's generalisation performance. Experimental results demonstrate that the authors’ proposed approach obtains significant performance on the two widely used benchmarks, that is, HICO‐DET and V‐COCO.

Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise

The open sourcing of large amounts of image data promotes the development of deep learning techniques. Along with this comes the privacy risk of these image datasets being exploited by unauthorized third parties to train deep learning models for commercial or illegal purposes. To avoid the abuse of data, a poisoning-based technique, "unlearnable example", has been proposed to significantly degrade the generalization performance of models by adding imperceptible noise to the data. To further enhance its robustness against adversarial training, existing works leverage iterative adversarial training on both the defensive noise and the surrogate model. However, it still remains unknown whether the robustness of unlearnable examples primarily comes from the effect of enhancement in the surrogate model or the defensive noise. Observing that simply removing the adversarial perturbation on the training process of the defensive noise can improve the performance of robust unlearnable examples, we identify that solely the surrogate model's robustness contributes to the performance. Furthermore, we found a negative correlation exists between the robustness of defensive noise and the protection performance, indicating defensive noise's instability issue. Motivated by this, to further boost the robust unlearnable example, we introduce Stable Error-Minimizing noise (SEM), which trains the defensive noise against random perturbation instead of the time-consuming adversarial perturbation to improve the stability of defensive noise. Through comprehensive experiments, we demonstrate that SEM achieves a new state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet Subset regarding both effectiveness and efficiency.

TDLC: Tensor decomposition‐based direct learning‐compression algorithm for DNN model compression

SummaryAs a deep neural networks (DNNs) model compression method, learning‐compression (LC) algorithm based on pre‐trained models and matrix decomposition increases training time and ignores the structural information of models. In this manuscript, a tensor decomposition‐based direct LC (TDLC) algorithm without pre‐trained models is proposed. In TDLC, the pre‐trained model is eliminated, and tensor decomposition is first applied to LC algorithm to preserve the structural features of the model. There are two key steps in TDLC. An optimal rank selection method is first proposed in compression‐step (C‐step) of TDLC to find global optimal ranks of tensor decomposition. Second, TDLC utilizes cyclical learning rate, which is different from traditional monotonically learning rates schedule, to improve the generalization performance of uncompressed models in learning‐step (L‐step). TDLC obtains the optimal compression model by alternately optimizing L‐step and C‐step. TDLC is compared with 16 state‐of‐the‐art compression methods in experiments part. Extensive experimental results show that TDLC produces high‐accuracy compression models with high compression rate. Comparing with TDLC‐pre‐trained, TDLC notably achieves 30% training time shorten and 11% parameter reduction in Resnet32, while improving accuracy by 0.2%.

Vocoder Detection of Spoofing Speech Based on GAN Fingerprints and Domain Generalization

As an important part of the text-to-speech (TTS) system, vocoders convert acoustic features into speech waveforms. The difference in vocoders is key to producing different types of forged speech in the TTS system. With the rapid development of general adversarial networks (GANs), an increasing number of GAN vocoders have been proposed. Detectors often encounter vocoders of unknown types, which leads to a decline in the generalization performance of models. However, existing studies lack research on detection generalization based on GAN vocoders. To solve this problem, this study proposes vocoder detection of spoofed speech based on GAN fingerprints and domain generalization. The framework can widen the distance between real speech and forged speech in feature space, improving the detection model’s performance. Specifically, we utilize a fingerprint extractor based on an autoencoder to extract GAN fingerprints from vocoders. We then weight them to the forged speech for subsequent classification to learn the forged speech features with high differentiation. Subsequently, domain generalization is used to further improve the generalization ability of the model for unseen forgery types. We achieve domain generalization using domain-adversarial learning and asymmetric triplet loss to learn a better generalized feature space in which real speech is compact and forged speech synthesized by different vocoders is dispersed. Finally, to optimize the training process, curriculum learning is used to dynamically adjust the contributions of the samples with different difficulties in the training process. Experimental results show that the proposed method achieves the most advanced detection results among four GAN vocoders. The code is available at https://github.com/multimedia-infomation-security/GAN-Vocoder-detection .

BiPSTP: Sequence feature encoding method for identifying different RNA modifications with bidirectional position-specific trinucleotides propensities

RNA modification, a post-transcriptional regulatory mechanism, significantly influences RNA biogenesis and function. The accurate identification of modification sites is paramount for investigating their biological implications. Methods for encoding RNA sequence into numerical data play a crucial role in developing robust models for predicting modification sites. However, existing techniques suffer from limitations, including inadequate information representation, challenges in effectively integrating positional and sequential information, and the generation of irrelevant or redundant features when combining multiple approaches. These deficiencies hinder the effectiveness of machine learning models in addressing the performance challenges associated with predicting RNA modification sites. Here we introduce a novel RNA sequence feature representation method, named BiPSTP, which utilizes bidirectional trinucleotide position-specific propensities. We employ the parameter ξ to denote the interval between the current nucleotide and its adjacent forward or backward dinucleotide, enabling the extraction of positional and sequential information from RNA sequences. Leveraging the BiPSTP method, we have developed the prediction model mRNAPred using support vector machine (SVM) classifier to identify multiple types of RNA modification sites. We evaluate the performance of our BiPSTP method and mRNAPred model across 12 distinct RNA modification types. Our experimental results demonstrate the superiority of the mRNAPred model compared to state-of-art models in the domain of RNA modification sites identification. Importantly, our BiPSTP method enhances the robustness and generalization performance of prediction models. Notably, it can be applied to feature extraction from DNA sequences to predict other biological modification sites.

A bidirectional interpretable compound-protein interaction prediction framework based on cross attention

The identification of compound-protein interactions (CPIs) plays a vital role in drug discovery. However, the huge cost and labor-intensive nature in vitro and vivo experiments make it urgent for researchers to develop novel CPI prediction methods. Despite emerging deep learning methods have achieved promising performance in CPI prediction, they also face ongoing challenges: (i) providing bidirectional interpretability from both the chemical and biological perspective for the prediction results; (ii) comprehensively evaluating model generalization performance; (iii) demonstrating the practical applicability of these models. To overcome the challenges posed by current deep learning methods, we propose a cross multi-head attention oriented bidirectional interpretable CPI prediction model (CmhAttCPI). First, CmhAttCPI takes molecular graphs and protein sequences as inputs, utilizing the GCW module to learn atom features and the CNN module to learn residue features, respectively. Second, the model applies cross multi-head attention module to compute attention weights for atoms and residues. Finally, CmhAttCPI employs a fully connected neural network to predict scores for CPIs. We evaluated the performance of CmhAttCPI on balanced datasets and imbalanced datasets. The results consistently show that CmhAttCPI outperforms multiple state-of-the-art methods. We constructed three scenarios based on compound and protein clustering and comprehensively evaluated the model generalization ability within these scenarios. The results demonstrate that the generalization ability of CmhAttCPI surpasses that of other models. Besides, the visualizations of attention weights reveal that CmhAttCPI provides chemical and biological interpretation for CPI prediction. Moreover, case studies confirm the practical applicability of CmhAttCPI in discovering anticancer candidates.

Effector-independent Representations Guide Sequential Target Selection Biases in Action.

Previous work shows that automatic attention biases toward recently selected target features transfer across action and perception and even across different effectors such as the eyes and hands on a trial-by-trial basis. Although these findings suggest a common neural representation of selection history across effectors, the extent to which information about recently selected target features is encoded in overlapping versus distinct brain regions is unknown. Using fMRI and a priming of pop-out task where participants selected unpredictable, uniquely colored targets among homogeneous distractors via reach or saccade, we show that color priming is driven by shared, effector-independent underlying representations of recent selection history. Consistent with previous work, we found that the intraparietal sulcus (IPS) was commonly activated on trials where target colors were switched relative to those where the colors were repeated; however, the dorsal anterior insula exhibited effector-specific activation related to color priming. Via multivoxel cross-classification analyses, we further demonstrate that fine-grained patterns of activity in both IPS and the medial temporal lobe encode information about selection history in an effector-independent manner, such that ROI-specific models trained on activity patterns during reach selection could predict whether a color was repeated or switched on the current trial during saccade selection and vice versa. Remarkably, model generalization performance in IPS and medial temporal lobe also tracked individual differences in behavioral priming sensitivity across both types of action. These results represent a first step to clarify the neural substrates of experience-driven selection biases in contexts that require the coordination of multiple actions.

A framework for demonstrating practical quantum advantage: comparing quantum against classical generative models

Generative modeling has seen a rising interest in both classical and quantum machine learning, and it represents a promising candidate to obtain a practical quantum advantage in the near term. In this study, we build over an existing framework for evaluating the generalization performance of generative models, and we establish the first quantitative comparative race towards practical quantum advantage (PQA) between classical and quantum generative models, namely Quantum Circuit Born Machines (QCBMs), Transformers (TFs), Recurrent Neural Networks (RNNs), Variational Autoencoders (VAEs), and Wasserstein Generative Adversarial Networks (WGANs). After defining four types of PQAs scenarios, we focus on what we refer to as potential PQA, aiming to compare quantum models with the best-known classical algorithms for the task at hand. We let the models race on a well-defined and application-relevant competition setting, where we illustrate and demonstrate our framework on 20 variables (qubits) generative modeling task. Our results suggest that QCBMs are more efficient in the data-limited regime than the other state-of-the-art classical generative models. Such a feature is highly desirable in a wide range of real-world applications where the available data is scarce.

Multiscale Change Detection Domain Adaptation Model Based on Illumination–Reflection Decoupling

In the change detection (CD) task, the substantial variation in feature distributions across different CD datasets significantly limits the reusability of supervised CD models. To alleviate this problem, we propose an illumination–reflection decoupled change detection multi-scale unsupervised domain adaptation model, referred to as IRD-CD-UDA. IRD-CD-UDA maintains its performance on the original dataset (source domain) and improves its performance on unlabeled datasets (target domain) through a novel CD-UDA structure and methodology. IRD-CD-UDA synergizes mid-level global feature marginal distribution domain alignment, classifier layer feature conditional distribution domain alignment, and an easy-to-hard sample selection strategy to increase the generalization performance of CD models on cross-domain datasets. Extensive experiments conducted on the LEVIR, SYSU, and GZ optical remote sensing image datasets demonstrate that the IRD-CD-UDA model effectively mitigates feature distribution discrepancies between source and target CD data, thereby achieving optimal recognition performance on unlabeled target domain datasets.

HCLR-Net: Hybrid Contrastive Learning Regularization with Locally Randomized Perturbation for Underwater Image Enhancement

AbstractUnderwater image enhancement presents a significant challenge due to the complex and diverse underwater environments that result in severe degradation phenomena such as light absorption, scattering, and color distortion. More importantly, obtaining paired training data for these scenarios is a challenging task, which further hinders the generalization performance of enhancement models. To address these issues, we propose a novel approach, the Hybrid Contrastive Learning Regularization (HCLR-Net). Our method is built upon a distinctive hybrid contrastive learning regularization strategy that incorporates a unique methodology for constructing negative samples. This approach enables the network to develop a more robust sample distribution. Notably, we utilize non-paired data for both positive and negative samples, with negative samples are innovatively reconstructed using local patch perturbations. This strategy overcomes the constraints of relying solely on paired data, boosting the model’s potential for generalization. The HCLR-Net also incorporates an Adaptive Hybrid Attention module and a Detail Repair Branch for effective feature extraction and texture detail restoration, respectively. Comprehensive experiments demonstrate the superiority of our method, which shows substantial improvements over several state-of-the-art methods in terms of quantitative metrics, significantly enhances the visual quality of underwater images, establishing its innovative and practical applicability. Our code is available at: https://github.com/zhoujingchun03/HCLR-Net.

Interpretable CNN-Multilevel Attention Transformer for Rapid Recognition of Pneumonia From Chest X-Ray Images.

Chest imaging plays an essential role in diagnosing and predicting patients with COVID-19 with evidence of worsening respiratory status. Many deep learning-based approaches for pneumonia recognition have been developed to enable computer-aided diagnosis. However, the long training and inference time makes them inflexible, and the lack of interpretability reduces their credibility in clinical medical practice. This paper aims to develop a pneumonia recognition framework with interpretability, which can understand the complex relationship between lung features and related diseases in chest X-ray (CXR) images to provide high-speed analytics support for medical practice. To reduce the computational complexity to accelerate the recognition process, a novel multi-level self-attention mechanism within Transformer has been proposed to accelerate convergence and emphasize the task-related feature regions. Moreover, a practical CXR image data augmentation has been adopted to address the scarcity of medical image data problems to boost the model's performance. The effectiveness of the proposed method has been demonstrated on the classic COVID-19 recognition task using the widespread pneumonia CXR image dataset. In addition, abundant ablation experiments validate the effectiveness and necessity of all of the components of the proposed method.

A double‐layer neural network wind speed prediction framework based on training set segmentation and error correction

AbstractHighly accurate wind speed prediction is one of the effective ways to improve wind energy utilization. Therefore, this study proposes a novel double‐layer neural network (D‐NN) framework based on feedforward neural networks. First, a feedforward neural network with an extremely short training time is used as the basic learner to ensure that the D‐NN framework has a short training and optimization time. Then, the double‐layer neural network automatically segments and adjusts the length of the training set using an improved error correction method, thus improving the model's ability to capture sequence features and enhancing the model's generalization performance in a non‐directly‐connected superposition manner. Finally, a genetic algorithm is used to automatically optimize the hyperparameters of each sub‐neural network in the D‐NN and the optimal training set length of each sub‐neural network, resulting in the final optimized prediction model. The accuracy and reliability of the prediction models designed based on the D‐NN framework are verified by comparing several different models. The results of the five different prediction evaluation metrics show that the D‐NN framework can greatly improve the accuracy of wind speed prediction.

Harnessing Generative Pre-Trained Transformers for Construction Accident Prediction with Saliency Visualization

Leveraging natural language processing models using a large volume of text data in the construction safety domain offers a unique opportunity to improve understanding of safety accidents and the ability to learn from them. However, little effort has been made to date in regard to utilizing large language models for the prediction of accident types that can help to prevent and manage potential accidents. This research aims to develop a model for predicting the six types of accidents (caught-in-between, cuts, falls, struck-by, trips, and others) by employing transfer learning with a fine-tuned generative pre-trained transformer (GPT). Additionally, to enhance the interpretability of the fine-tuned GPT model, a method for saliency visualization of input text was developed to identify words that significantly impact prediction results. The models were evaluated using a comprehensive dataset comprising 15,000 actual accident records. The results indicate that the suggested model for detecting the six accident types achieves 82% accuracy. Furthermore, it was observed that the proposed saliency visualization method can identify accident precursors from unstructured free-text data of construction accident reports. These results highlight the advancement of the generalization performance of large language processing-based accident prediction models, thereby proactively preventing construction accidents.

A novel ensemble learning method for crop leaf disease recognition.

Deep learning models have been widely applied in the field of crop disease recognition. There are various types of crops and diseases, each potentially possessing distinct and effective features. This brings a great challenge to the generalization performance of recognition models and makes it very difficult to build a unified model capable of achieving optimal recognition performance on all kinds of crops and diseases. In order to solve this problem, we have proposed a novel ensemble learning method for crop leaf disease recognition (named ELCDR). Unlike the traditional voting strategy of ensemble learning, ELCDR assigns different weights to the models based on their feature extraction performance during ensemble learning. In ELCDR, the models' feature extraction performance is measured by the distribution of the feature vectors of the training set. If a model could distinguish more feature differences between different categories, then it receives a higher weight during ensemble learning. We conducted experiments on the disease images of four kinds of crops. The experimental results show that in comparison to the optimal single model recognition method, ELCDR improves by as much as 1.5 (apple), 0.88 (corn), 2.25 (grape), and 1.5 (rice) percentage points in accuracy. Compared with the voting strategy of ensemble learning, ELCDR improves by as much as 1.75 (apple), 1.25 (corn), 0.75 (grape), and 7 (rice) percentage points in accuracy in each case. Additionally, ELCDR also has improvements on precision, recall, and F1 measure metrics. These experiments provide evidence of the effectiveness of ELCDR in the realm of crop leaf disease recognition.

A novel AI-driven model for erosion prediction for elbow in gas-solid two-phase flows

Erosion is inevitable in pipeline gathering and transportation systems. Predicting erosion in typical pipe fittings is pivotal to ensure the safety and efficiency of the equipment. This work develops a novel predictive model for erosion in elbows under gas-solid flow conditions. The proposed model employs the Deep Belief Network (DBN) for feature extraction from the input parameters. Meanwhile, Kernel Extreme Learning Machine (KELM) is implemented to deeply analyze correlation between the eigenvalues and the erosion rate, thus improving the model's generalization performance. Furthermore, the Improved Coati Optimization Algorithm (ICOA) is utilized to optimize the model's hyperparameters, contributing to increased accuracy. To validate the reliability of the ICOA - DBN - KELM model, it was applied to both the experimental and simulated datasets. The outcomes demonstrate that the proposed model attains the highest prediction precision in both datasets compared to other models. Additionally, the model was assessed for reasonableness concerning trend consistency and dimensionless numbers.

Efficient first principles based modeling via machine learning: from simple representations to high entropy materials

Generalization performance of machine learning models: (upper panel) generalization from small ordered to large disordered structures (SQS); (lower panel) generalization from low-order to high-order systems.

EMG-based Multi-User Hand Gesture Classification via Unsupervised Transfer Learning Using Unknown Calibration Gestures.

The poor generalization performance and heavy training burden of the gesture classification model contribute as two main barriers that hinder the commercialization of sEMG-based human-machine interaction (HMI) systems. To overcome these challenges, eight unsupervised transfer learning (TL) algorithms developed on the basis of convolutional neural networks (CNNs) were explored and compared on a dataset consisting of 10 gestures from 35 subjects. The highest classification accuracy obtained by CORrelation Alignment (CORAL) reaches more than 90%, which is 10% higher than the methods without using TL. In addition, the proposed model outperforms 4 common traditional classifiers (KNN, LDA, SVM, and Random Forest) using the minimal calibration data (two repeated trials for each gesture). The results also demonstrate the model has a great transfer robustness/flexibility for cross-gesture and cross-day scenarios, with an accuracy of 87.94% achieved using calibration gestures that are different with model training, and an accuracy of 84.26% achieved using calibration data collected on a different day, respectively. As the outcomes confirm, the proposed CNN TL method provides a practical solution for freeing new users from the complicated acquisition paradigm in the calibration process before using sEMG-based HMI systems.